1. History and General Statement of the Problem
In 1984, the National Highway Traffic Safety Administration (NHTSA) of the U.S. Department of Transportation issued a requirement for frontal crash protection of automobile occupants. This regulation mandated “passive occupant restraints” for all passenger cars by 1992. A more recent regulation required both driver and passenger side airbags on all passenger cars and light trucks by 1998. In addition, the demand for airbags is constantly accelerating in both Europe and Japan (˜36 million vehicles) and all vehicles produced in these areas and eventually worldwide (˜50 million vehicles) will likely be equipped with airbags as standard equipment, if they do not include them already.
Whereas thousands of lives have been saved by airbags, significant improvements can be made. As discussed in detail in U.S. Pat. No. 5,653,462 referenced above, for a variety of reasons, vehicle occupants can be or get too close to the airbag before it deploys and can be seriously injured or killed upon deployment of the airbag.
Also, a child in a rear facing child seat, which is placed on the right front passenger seat, is in danger of being seriously injured if the passenger airbag deploys. This has now become an industry-wide concern and the U.S. automobile industry is continually searching for an easy, economical solution, which will prevent the deployment of the passenger side airbag if a rear facing child seat is present. An improvement on the invention disclosed in the above-referenced patent, as will be disclosed in greater detail below, includes more sophisticated means to identify objects within the passenger compartment and will solve this problem.
Initially, these systems will solve the out-of-position occupant and the rear facing child seat problems related to current airbag systems and prevent unneeded deployments when a seat is unoccupied. Airbags are now under development to protect rear seat occupants in vehicle crashes. A system is therefore needed to detect the presence of occupants, position, (to determine if they are out-of-position), and type(to identify the presence of a rear facing child seat) in the rear seat. Current and future automobiles may have eight or more airbags as protection is sought for rear seat occupants and from side impacts. In addition to eliminating the disturbance of unnecessary airbag deployments, the cost of replacing these airbags will be excessive if they all deploy in an accident. The improvements described below minimize this cost by not deploying an airbag for a seat, which is not occupied by a human being. An occupying item of a seat may be a living occupant such as a human being or dog, another living organism such as a plant, or an inanimate object such as a box or bag of groceries.
Side impact airbag systems began appearing on 1995 vehicles. The danger of deployment induced injuries will exist for side impact airbags as they now do for frontal impact airbags. A child with his head against the airbag is such an example. The system of this invention will minimize such injuries.
2. General Solution to the Problem and Resulting Benefits
A device to monitor the vehicle interior and identify its contents is needed to solve these and many other related problems. For example, once a Vehicle Interior Identification and Monitoring System (VIMS) for identifying and monitoring the contents of a vehicle is in place, many other products become possible including the following:
Inflators and control systems now exist which will adjust the amount of gas flowing into and/or out of the airbag to account for the size and position of the occupant and for the severity of the accident. The vehicle identification and monitoring system of this invention will control such systems based on the presence, size and position of vehicle occupants or the presence, position and orientation of an occupied child seat.
Vehicles can be provided with a standard cellular phone or other telematics communication system as well as the Global Positioning System (GPS), an automobile navigation or location system with an optional connection to a manned assistance facility, which is now available on several vehicle models. In the event of an accident, the phone may automatically call 911, or contact OnStar™ or similar service for emergency assistance and report the position of the vehicle. If the vehicle also has a system as described below for monitoring each seat location, the number and perhaps the condition of the occupants could also be reported and/or photographs of the vehicle interior before, during and/or after the accident can be transmitted. In that way, the emergency service (EMS) would know what equipment and how many ambulances to send to the accident site and prioritize the accident when several accidents occur in the same time frame. Moreover, a communication channel can be opened between the vehicle and a monitoring facility/emergency response facility or personnel to enable directions to be provided to the injured occupant(s) of the vehicle to assist in any necessary first aid prior to arrival of the emergency assistance personnel.
Once an occupant sensor is operational, the vehicle entertainment system can be improved if the number, size and location of occupants and other objects were known. However, it is not believed that, prior to the instant invention, engineers have thought to determine the number, size and/or location of the occupants and use such determination in combination with the entertainment system. Indeed, this information can be provided by the vehicle interior identification and monitoring system of this invention to thereby improve a vehicle's entertainment system. Once one considers monitoring the space in the passenger compartment an alternate method of characterizing the sonic environment comes to mind which is to send and receive a test sound to see what frequencies are reflected, absorbed or excite resonances and then adjust the spectral output of the entertainment system accordingly.
As the VIMS improves to where such things as the exact location of the occupants ears and eyes can be determined, even more significant improvements to the entertainment system become possible through the use of noise canceling sound, and the rear view mirrors can be automatically adjusted for the driver's eye location.
Another example involves the monitoring of the driver's behavior over time that can be used to warn a driver if he or she is falling asleep, or to stop the vehicle if the driver loses the capacity to control it.
Similarly to the entertainment system, the heating, ventilation and air conditioning system (HVAC) can be improved if the number, attributes and location of vehicle occupants were known. This can be used to provide a climate control system tailored to each occupant, for example, or the system can be turned off for certain seat locations if there are no occupants present at those locations.
In some cases, the position of a particular part of the occupant is of interest such as: (a) his hand or arm and whether it is in the path of a closing window so that the motion of the window needs to be stopped; (b) the position of the shoulder so that the seat belt anchorage point can be adjusted for the best protection of the occupant; or, (c) the position of the rear of the occupants head so that the headrest can be adjusted to minimize whiplash injuries in rear impacts.
Additionally, using an advanced VIMS, as explained below, the position of the driver's eyes can be accurately determined and portions of the windshield, or of a special visor, can be selectively darkened to eliminate the glare from the sun or oncoming vehicle headlights. This system can use electro-chromic glass, a liquid crystal device, Xerox Gyricon, Research Frontiers SPD, semiconducting and metallic (organic) polymer displays, spatial light monitors, electronic “Venetian blinds”, electronic polarizers or other appropriate technology, and, in some cases, detectors to detect the direction of the offending light source. In addition to eliminating the glare, the standard sun visor can now also be eliminated. Alternately, the glare filter can be placed in another device such as a transparent sun visor that is placed between the driver's eyes and the windshield.
3. Pattern Recognition
The present invention adds more sophisticated pattern recognition capabilities such as fuzzy logic systems, neural networks, modular neural network systems or other pattern recognition computer based algorithms to the occupant position measurement system disclosed in the above referenced patents and/or patent applications and greatly extends the areas of application of this technology. An example of such a pattern recognition system using neural networks using sonar is discussed in two papers by Gorman, R. P. and Sejnowski, T. J. “Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets”, Neural Networks, Vol.1. pp. 75–89, 1988, and “Learned Classification of Sonar Targets Using a Massively Parallel Network”, IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 36, No. 7, July 1988.
4. Definitions
Some embodiments of the invention are described below and unless specifically noted, it is the applicants' intention that the words and phrases in the specification and claims be given the ordinary and accustomed meaning to those of ordinary skill in the applicable art(s). If the applicant intends any other meaning, he will specifically state he is applying a special meaning to a word or phrase.
Likewise, applicants' use of the word “function” here is not intended to indicate that the applicants seek to invoke the special provisions of 35 U.S.C. §112, sixth paragraph, to define their invention. To the contrary, if applicants wish to invoke the provisions of 35 U.S.C.§112, sixth paragraph, to define their invention, they will specifically set forth in the claims the phrases “means for” or “step for” and a function, without also reciting in that phrase any structure, material or act in support of the function. Moreover, even if applicants invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, it is the applicants' intention that their inventions not be limited to the specific structure, material or acts that are described in the preferred embodiments herein. Rather, if applicants claim their inventions by specifically invoking the provisions of 35 U.S.C. §112, sixth paragraph, it is nonetheless their intention to cover and include any and all structure, materials or acts that perform the claimed function, along with any and all known or later developed equivalent structures, materials or acts for performing the claimed function.
“Pattern recognition” as used herein will generally mean any system which processes a signal that is generated by an object, or is modified by interacting with an object, in order to determine which one of a set of classes that the object belongs to. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally electrical signals coming from transducers which are sensitive to either acoustic or electromagnetic radiation and if electromagnetic, they can be either visible light, infrared, ultraviolet, radar or other part of the electromagnetic spectrum, or electric or magnetic fields.
“To identify” as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing all rear facing child seats, one containing all human occupants, all human occupants not sitting in a rear facing child seat, or all humans in a certain height or weight range, all humans that are in a position where they can be protected by an airbag, all humans that are in a position where they are at risk to be seriously injured by an airbag depending on the purpose of the system. In the case where a particular person is to be recognized, the set or class will contain only a single element, the person to be recognized.
5. Some Examples of the Invention
In a passive infrared system a detector receives infrared radiation from an object in its field of view, in this case the vehicle occupant, and determines the temperature of the occupant based on the infrared radiation. The VIMS can then respond to the temperature of the occupant, which can either be a child in a rear facing child seat or a normally seated occupant, to control some other system. This technology can provide input data to a pattern recognition system but it has limitations related to temperature. The sensing of the child could pose a problem if the child is covered with blankets. It also might not be possible to differentiate between a rear facing child seat and a forward facing child seat. In all cases, the technology will fail to detect the occupant if the ambient temperature reaches body temperature as it does in hot climates. Nevertheless, for use in the control of the vehicle climate, for example, a passive infrared system that permits an accurate measurement of each occupant's temperature is useful.
In a laser optical system an infrared laser beam is used to momentarily illuminate an object, occupant or child seat in the manner as described, and illustrated in FIG. 8, of U.S. Pat. No. 5,653,462 cross-referenced above. In some cases, a charge-coupled device (a type of TV camera also referred to as a CCD array) or a CMOS device is used to receive the reflected light. In other cases when a scanning laser is used a pin or avalanche diode or other photo detector can be used. The laser can either be used in a scanning mode, or, through the use of a lens, a cone of light, swept line of light, or a pattern or structured light can be created which covers a large portion of the object. Additionally, one or more LEDs can be used as a light source.
Also, triangulation can be used in conjunction with an offset scanning laser to determine the range of the illuminated spot from the light detector. Various focusing systems also can have applicability in some implementations to measure the distance to an occupant. In most cases, a pattern recognition system, as defined above, is used to identify and classify, and can be used to locate, the illuminated object and its constituent parts. The optical systems generally provide the most information about the object and at a rapid data rate. Its main drawback is cost which is considerably above that of ultrasonic or passive infrared systems. As the cost of lasers and imagers comes down in the future, this system will become more competitive. Depending on the implementation of the system, there may be some concern for the safety of the occupant if a laser light can enter the occupant's eyes. This is minimized if the laser operates in the infrared spectrum particularly at the “eye-safe” frequencies.
Radar systems have similar properties to the laser system discussed above. The wavelength of a particular radar system can limit the ability of the pattern recognition system to detect object features smaller than a certain size. Also, depending on the radar frequency, the detecting method can be based on the modification of the waves in different ways such as reflection, absorption, scattering or transmission. Once again, however, there is some concern about the health effects of radar on children and other occupants. This concern is expressed in various reports available from the United States Food and Drug Administration Division of Devices.
Naturally, electromagnetic waves from other parts of the electromagnetic spectrum can also be used such as, for example, those used with what are sometimes referred to as capacitive or electric field sensors, for example as described in U.S. patents by Kithil et al. Nos. 5,366,241, 5,602,734, 5,691,693, 5,802,479, 5,844,486 and 6,014,602; by Jinno et al. U.S. Pat. No. 5,948,031; and SAE technical papers 982292 and 971051 which are incorporated herein by reference. Additionally, as discussed in more detail below, the sensing of the change in the characteristics of the near field that surrounds an antenna is an effective and economical method of determining the presence of water in the vicinity of the antenna and thus a measure of occupant presence. The use of electric field and capacitance sensors and their equivalence to the occupant sensors described herein requires a special discussion.
6. Electric Field, Capacitance and Wave Sensors
Electric field sensors and wave sensors are essentially the same from the point of view of sensing the presence of an occupant in a vehicle. In both cases, a time varying electric field is disturbed or modified by the presence of the occupant. At high frequencies in the visual, infrared and high frequency radio wave region, the sensor is based on its capability to sense change of wave characteristics of the electromagnetic field, such as amplitude, phase or frequency. As the frequency drops, other characteristics of the field are measured. At still lower frequencies, the occupant's dielectric properties modify parameters of the reactive electric field in the occupied space between/near the plates of a capacitor. In this latter case, the sensor senses the change in charge distribution on the capacitor plates by measuring, for example, the current wave magnitude or phase in the electric circuit that drives the capacitor. These measured parameters are directly connected with parameters of the displacement current in the occupied space. In all cases, the presence of the occupant reflects, absorbs or modifies the waves or variations in the electric field in the space occupied by the occupant. Thus for the purposes of this invention, capacitance, electric field or electromagnetic wave sensors are equivalent and although they are all technically “field” sensors they will be considered as “wave” sensors herein. What follows is a discussion comparing the similarities and differences between two types of field or wave sensors, electromagnetic wave sensors and capacitive sensors as exemplified by Kithil in U.S. Pat. No. 5,602,734.
An electromagnetic field disturbed or emitted by a passenger in the case of an electromagnetic wave sensor, for example, and the electric field sensor of Kithil, for example, are in many ways similar and equivalent for the purposes of this invention. The electromagnetic wave sensor is an actual electromagnetic wave sensor by definition because they sense parameters of a wave, which is a coupled pair of continuously changing electric and magnetic fields. The electric field here is not a static, potential one. It is essentially a dynamic, rotational electric field coupled with a changing magnetic one, that is, an electromagnetic wave. It cannot be produced by a steady distribution of electric charges. It is initially produced by moving electric charges in a transmitter, even if this transmitter is a passenger body for the case of a passive infrared sensor.
In the Kithil sensor, a static electric field is declared as an initial material agent coupling a passenger and a sensor (see Column 5, lines 5–7): “The proximity sensor 12 each function by creating an electrostatic field between oscillator input loop 54 and detector output loop 56, which is affected by presence of a person near by, as a result of capacitive coupling, . . . ”). It is a potential, non-rotational electric field. It is not necessarily coupled with any magnetic field. It is the electric field of a capacitor. It can be produced with a steady distribution of electric charges. Thus, it is not an electromagnetic wave by definition but if the sensor is driven by a varying current, then it produces a quasistatic electric field in the space between/near the plates of the capacitor.
Kithil declares that he uses a static electric field in his capacitance sensor. Thus, from the consideration above, one can conclude that Kithil's sensor cannot be treated as a wave sensor because there are no actual electromagnetic waves but only a static electric field of the capacitor in the sensor system. However, this is not believed to be the case. The Kithil system could not operate with a true static electric field because a steady system does not carry any information. Therefore, Kithil is forced to use an oscillator, causing an alternate current in the capacitor and a reactive quasistatic electric field in the space between the capacitor plates, and a detector to reveal an informative change of the sensor capacitance caused by the presence of an occupant (see FIG. 7 and its description). In this case, the system becomes a “wave sensor” in the sense that it starts generating actual time-varying electric field that certainly originates electromagnetic waves according to the definition above. That is, Kithil's sensor can be treated as a wave sensor regardless of the shape of the electric field that it creates, a beam or a spread shape.
As follows from the Kithil patent, the capacitor sensor is likely a parametric system where the capacitance of the sensor is controlled by influence of the passenger body. This influence is transferred by means of the near electromagnetic field (i.e., the wave-like process) coupling the capacitor electrodes and the body. It is important to note that the same influence takes place with a real static electric field also, that is in absence of any wave phenomenon. This would be a situation if there were no oscillator in Kithil's system. However, such a system is not workable and thus Kithil reverts to a dynamic system using time-varying electric fields.
Thus, although Kithil declares the coupling is due to a static electric field, such a situation is not realized in his system because an alternating electromagnetic field (“quasi-wave”) exists in the system due to the oscillator. Thus, his sensor is actually a wave sensor, that is, it is sensitive to a change of a wave field in the vehicle compartment. This change is measured by measuring the change of its capacitance. The capacitance of the sensor system is determined by the configuration of its electrodes, one of which is a human body, that is, the passenger inside of and the part which controls the electrode configuration and hence a sensor parameter, the capacitance.
The physics definition of “wave” from Webster's Encyclopedic Unabridged Dictionary is:“ 11. Physics. A progressive disturbance propagated from point to point in a medium or space without progress or advance of the points themselves, . . . ”. In a capacitor, the time that it takes for the disturbance (a change in voltage) to propagate through space, the dielectric and to the opposite plate is generally small and neglected but it is not zero. As the frequency driving the capacitor increases and the distance separating the plates increases, this transmission time as a percentage of the period of oscillation can become significant. Nevertheless, an observer between the plates will see the rise and fall of the electric field much like a person standing in the water of an ocean. The presence of a dielectric body between the plates causes the waves to get bigger as more electrons flow to and from the plates of the capacitor. Thus, an occupant affects the magnitude of these waves which is sensed by the capacitor circuit. Thus, the electromagnetic field is a material agent that carries information about a passenger's position in both Kithil's and a beam-type electromagnetic wave sensor.
The following definitions are applicable and are from the Encyclopedia Britannica:
“electromagnetic field”
“A property of space caused by the motion of an electric charge. A stationary charge will produce only an electric field in the surrounding space. If the charge is moving, a magnetic field is also produced. An electric field can be produced also by a changing magnetic field. The mutual interaction of electric and magnetic fields produces an electromagnetic field, which is considered as having its own existence in space apart from the charges or currents (a stream of moving charges) with which it may be related . . . ” (Copyright 1994–1998 Encyclopedia Britannica)
“displacement current”
“. . . in electromagnetism, a phenomenon analogous to an ordinary electric current, posited to explain magnetic fields that are produced by changing electric fields. Ordinary electric currents, called conduction currents, whether steady or varying, produce an accompanying magnetic field in the vicinity of the current. [. . . ]
“As electric charges do not flow through the insulation from one plate of a capacitor to the other, there is no conduction current; instead, a displacement current is said to be present to account for the continuity of the magnetic effects. In fact, the calculated size of the displacement current between the plates of a capacitor being charged and discharged in an alternating-current circuit is equal to the size of the conduction current in the wires leading to and from the capacitor. Displacement currents play a central role in the propagation of electromagnetic radiation, such as light and radio waves, through empty space. A traveling, varying magnetic field is everywhere associated with a periodically changing electric field that may be conceived in terms of a displacement current. Maxwell's insight on displacement current, therefore, made it possible to understand electromagnetic waves as being propagated through space completely detached from electric currents in conductors.” Copyright 1994–1998 Encyclopedia Britannica
“electromagnetic radiation”
“. . . energy that is propagated through free space or through a material medium in the form of electromagnetic waves, such as radio waves, visible light, and gamma rays. The term also refers to the emission and transmission of such radiant energy. [. . . ]
“It has been established that time-varying electric fields can induce magnetic fields and that time-varying magnetic fields can in like manner induce electric fields. Because such electric and magnetic fields generate each other, they occur jointly, and together they propagate as electromagnetic waves. An electromagnetic wave is a transverse wave in that the electric field and the magnetic field at any point and time in the wave are perpendicular to each other as well as to the direction of propagation. [. . . ]
“Electromagnetic radiation has properties in common with other forms of waves such as reflection, refraction, diffraction, and interference. [. . . ]” Copyright 1994–1998 Encyclopedia Britannica
The main part of the Kithil “circuit means” is an oscillator, which is as necessary in the system as the capacitor itself to make the capacitive coupling effect be detectable. An oscillator by nature creates a time varying electric field in a capacitor or waves. The system can operate as a sensor only if an alternating current flows through the sensor capacitor, which, in fact, is a detector from which an informative signal is acquired. Then this current (or, more exactly, integral of the current over time—charge) is measured and the result is a measure of the sensor capacitance value. The latter in turn depends on the passenger presence that affects the magnitude of the waves that travel between the plates of the capacitor making the Kithil sensor a wave sensor by the definition herein.
An additional relevant definition is:
“Capacitive coupling”
The transfer of energy from one circuit to another by means of the mutual capacitance between the circuits. (188) The coupling may be deliberate or inadvertent. Capacitive coupling favors transfer of the higher frequency components of a signal, whereas inductive coupling favors lower frequency components, and conductive coupling favors neither higher nor lower frequency components.” http:// - - - .its.bldrdoc.gov/fs-1037/dir-006 /—0842.htm
Another similarity between one embodiment of the sensor of this invention and the Kithil sensor is the use of a voltage-controlled oscillator (VCO).
7. Ultrasonic Occupant Sensors
The ultrasonic system is the least expensive and potentially provides less information than the optical or some radar systems due to the delays resulting from the speed of sound and due to the wave length which is considerably longer than the optical (including infrared) systems. The longer wavelength limits the detail, which can be seen by the system. In spite of these limitations, as shown below, ultrasonics can provide sufficient timely information to permit the position and velocity of an occupant to be accurately known and, when used with an appropriate pattern recognition system, it is capable of positively determining the presence of a rear facing child seat. One pattern recognition system that has been used to identify a rear facing child seat is a neural networks and is similar to that described in the above-referenced papers by Gorman et al.
8. Occupant Location Based on Focusing
A focusing system, such as used on some camera systems, can be used to determine the initial position of an occupant but, in most cases, it is too slow to monitor his position during a crash. This is a result of the mechanical motions required to operate the lens focusing system, however, methods do exist that do not require mechanical motions. By itself it cannot determine the presence of a rear facing child seat or of an occupant but when used with a charge-coupled or CMSO device plus some infrared illumination for vision at night, and an appropriate pattern recognition system, this becomes possible. Similarly, the use of three dimensional cameras based on modulated waves or range-gated pulsed light methods combined with pattern recognition systems are now possible based on the teachings of the inventions disclosed herein and the commonly assigned patents and patent applications referenced above.
9. Information About an Occupying Item
As discussed above, it is desirable to obtain information about an occupying item in a vehicle in order to control a component in the vehicle based on the characteristics of the occupying item. For example, if it were known that the occupying item is inanimate, an airbag deployment system would generally be controlled to suppress deployment of any airbags designed to protect passengers seated at the location of the inanimate object.
When the occupying item is human, in some instances the information about the occupying item can be the occupant's position, size and weight. Each of these properties can have an effect on the control criteria of the component. One system for determining a deployment force of an air bag system in described in U.S. Pat. No. 6,199,904 (Dosdall). This system provides a reflective surface in the vehicle seat that reflects microwaves transmitted from a microwave emitter. The position, size and weight of a human occupant are said to be determined by calibrating the microwaves detected by a detector after the microwaves have been reflected from the reflective surface and pass through the occupant.
10. Child Seat Detection Prior Art
With respect to prior art related to the detection of child restraining seats, U.S. Pat. No. 5,605,348 (Blackburn et al.) describes method and apparatus for sensing a rearward facing child restraining seat in which a child restraining seat identification tag is secured to the child restraining seat and an antenna coil is energized to transmit an EMF field. The tag is made of an amorphous material that radiates a return EMF signal that is received by the antenna coil. The system determines whether a rear-facing child seat is present based on the presence of the return EMF signal, which is received only if the tag mounted to bottom front of the child seat is within a certain distance from the antenna coil mounted in the back portion of the seat.
Drawbacks of the system of Blackburn et al. are that a special tag must be incorporated into the child seat in order to detect the same, the system cannot differentiate between other similarly tagged objects and the system relies on the proper placement of the tag on the child seat. In other words, if the tag were to be improperly placed on the child seat, then the system would not accurately determine the presence and orientation of the same. Also, the system of Blackburn et al. does not generate, e.g., via the antenna coil, a signal based on the contents of the seat which is different depending on the contents of the seat, i.e., the signal for an adult occupant is different from the signal for a forward-facing child seat which is different from the signal for a rear-facing child seat, etc., and analyze the same in order to determine whether the contents of the seat include a child seat in a rear-facing position. Rather, the EMF field generated by the tag is the same signal, and only the power output is varied. Thus, either no signal (no EMF field) will be generated representing the absence of a rear-facing child seat or a signal (an EMF field) will be generated representing the presence of a rear-facing child seat.
11. Summary
From the above discussion, it can be seen that the addition of sophisticated pattern recognition means to any of the illumination and/or reception technologies for use in a motor vehicle permits the development of a host of new products, systems or capabilities heretofore not available and as described in more detail below.